故障诊断在许多领域至关重要，因为故障可能导致安全威胁或经济损失。在在线服务系统领域中，操作员依靠大量监视数据来检测和减轻故障。快速识别一组基础故障的根本原因指标可以节省大量时间减轻故障。在本文中，我们将根本原因分析问题作为一种新的因果推理任务，称为干预识别。我们提出了一种新型的无监督因果推理的方法，名为基于因果推理的根本原因分析（大约）。核心思想是一个足够的条件，可以使监视变量成为根本原因指标，即，因果关系贝叶斯网络（CBN）中父母的概率分布的变化。在在线服务系统中的应用程序中，大约根据系统体系结构的知识和一组因果假设在监视指标中构建图形。仿真研究说明了大约的理论可靠性。现实世界中数据集的性能进一步表明，大约可以将TOP-1建议的回忆提高到最佳基线方法的25％。

具有变压器（DETR）和相关工程的对象检测达到或甚至超越了具有自我关注网络架构的高度优化的更优化的rcnn基线。灵感灵感来自纯粹的自我关注具有强烈的电感偏差，导致变压器对网络深度失去表现力的能力，我们通过在变压器中应用可能的直接映射连接来提出具有减轻自我关注机制的变压器架构建筑以缓解排名崩溃，以抵消特征表达式损失并增强模型性能。我们在对象检测任务中应用此提议，并开发名为Miti-Detr的型号。 MITI-DETR将每个单一注意层的输入保留到该层的输出，以便“非关注”信息参与了任何关注传播。形成的残余自我关注网络解决了两个关键问题：（1）停止自我关注网络从退化到秩1到最大程度; （2）进一步多样化参数更新的路径分布，以便预期更容易学习。 MITI-DETR显着提高了在挑战COCO对象检测数据集上实现了现有DETR基模型的平均检测精度和收敛速度。此外，具有剩余自我关注网络的所提出的变压器可以在没有特定定制的情况下容易地概括或插入其他相关任务模型。

本文通过模仿人脑的学习和思维过程来提出基于语义聚类的扣除学习。人类可以根据经验和认知做出判决，结果，没有人会识别一个未知的动物作为汽车。灵感来自这种观察，我们建议使用之前的聚类培训深度学习模型，可以指导模型来学习语义的能力，从分类属性中宣传和总结，例如属于动物的猫而与车辆有关的汽车。特别是，如果图像被标记为猫，则培训模型以了解“此图像完全不是动物的异常值”。所提出的方法实现了语义空间中的高级聚类，使模型能够在学习过程中推断各种类之间的关系。此外，本文介绍了一种基于语义的基于语义的随机搜索，对相反的标签，以确保聚类的平滑分布和分类器的鲁棒性。理论上和经验通过广泛的实验支持拟议的方法。我们将跨新型分类器的性能进行比较，在流行的基准上，通过向数据集添加噪声标记来验证泛化能力。实验结果表明了所提出的方法的优越性。

精确分割牙齿并识别牙科网格模型上的相应解剖标签在计算机辅助性正畸治疗中是必不可少的。手动执行这两个任务是耗时，繁琐的，更重要的是，由于患者牙齿的异常和大规模差异，高度依赖于矫正者的经验。一些基于机器学习的方法已经设计和应用于正畸场，以自动分割牙科网格（例如，口腔扫描）。相比之下，牙齿地标定位的研究数量仍然有限。本文提出了一种基于网格深度学习（称为TS-MDL）的两级框架，用于联合牙齿标签和原始内部扫描的地标识别。我们的TS-MDL首先采用端到端\ EMPH {i} MeshsegNet方法（即，现有网格孔的变体，具有改进的精度和效率），以在下采样扫描上标记每个牙齿。由分割输出引导，我们的TS-MDL进一步选择原始网格上的每个牙齿的感兴趣区域（ROI），以构造开头的光重变量（即PINTNET-REG），用于回归相应的地标热插块。我们的TS-MDL在实际的数据集上进行了评估，显示了有希望的细分和本地化性能。具体而言，TS-MDL的第一阶段中的\ EMPH {i} Meshsegnet达到了0.964 \ PM0.054 $ 0.964 \ PM0.054 $的平均骰子相似度系数（DSC），显着优于原始的Meshsegnet。在第二阶段，PointNet-Reg实现了0.597 \ PM0.761 \，预测和地面真理之间的平均绝对误差（MAE），以66美元的地标，与地标检测的其他网络相比，比较优越。所有这些结果表明我们在临床实践中的TS-MDL潜在使用。

Masked image modeling (MIM) performs strongly in pre-training large vision Transformers (ViTs). However, small models that are critical for real-world applications cannot or only marginally benefit from this pre-training approach. In this paper, we explore distillation techniques to transfer the success of large MIM-based pre-trained models to smaller ones. We systematically study different options in the distillation framework, including distilling targets, losses, input, network regularization, sequential distillation, etc, revealing that: 1) Distilling token relations is more effective than CLS token- and feature-based distillation; 2) An intermediate layer of the teacher network as target perform better than that using the last layer when the depth of the student mismatches that of the teacher; 3) Weak regularization is preferred; etc. With these findings, we achieve significant fine-tuning accuracy improvements over the scratch MIM pre-training on ImageNet-1K classification, using all the ViT-Tiny, ViT-Small, and ViT-base models, with +4.2%/+2.4%/+1.4% gains, respectively. Our TinyMIM model of base size achieves 52.2 mIoU in AE20K semantic segmentation, which is +4.1 higher than the MAE baseline. Our TinyMIM model of tiny size achieves 79.6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget. This strong performance suggests an alternative way for developing small vision Transformer models, that is, by exploring better training methods rather than introducing inductive biases into architectures as in most previous works. Code is available at https://github.com/OliverRensu/TinyMIM.

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.

Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.

Few Shot Instance Segmentation (FSIS) requires models to detect and segment novel classes with limited several support examples. In this work, we explore a simple yet unified solution for FSIS as well as its incremental variants, and introduce a new framework named Reference Twice (RefT) to fully explore the relationship between support/query features based on a Transformer-like framework. Our key insights are two folds: Firstly, with the aid of support masks, we can generate dynamic class centers more appropriately to re-weight query features. Secondly, we find that support object queries have already encoded key factors after base training. In this way, the query features can be enhanced twice from two aspects, i.e., feature-level and instance-level. In particular, we firstly design a mask-based dynamic weighting module to enhance support features and then propose to link object queries for better calibration via cross-attention. After the above steps, the novel classes can be improved significantly over our strong baseline. Additionally, our new framework can be easily extended to incremental FSIS with minor modification. When benchmarking results on the COCO dataset for FSIS, gFSIS, and iFSIS settings, our method achieves a competitive performance compared to existing approaches across different shots, e.g., we boost nAP by noticeable +8.2/+9.4 over the current state-of-the-art FSIS method for 10/30-shot. We further demonstrate the superiority of our approach on Few Shot Object Detection. Code and model will be available.